On the Generalization Ability of Data-Driven Models in the Problem of Total Cloud Cover Retrieval

Total Cloud Cover (TCC) retrieval from ground-based optical imagery is a problem that has been tackled by several generations of researchers. The number human-designed algorithms for the estimation TCC grows every year. However, there no considerable progress in terms quality, mostly due to lack systematic approach design algorithms, assessment their generalization ability, and quality. In this study, we discuss optimization nature data-driven schemes retrieval. order compare propose framework algorithms’ characteristics. We present new are based on deep learning techniques: A model outliers filtering, few models all-sky imagery. For training Dataset All-Sky Imagery over Ocean (DASIO) containing one million images visible sky dome taken various regions world ocean. research campaigns contributed DASIO collection took place Atlantic ocean, Indian Red Mediterranean seas, Arctic Optical collected during these missions accompanied standard meteorological observations cloudiness characteristics made experienced observers. assess ability presented scenarios differ selected train test subsets. As result, demonstrate our convolutional neural networks deliver superior quality compared all previously published approaches. key drop generalize data case strong covariate shift between subsets which may occur region-aware subsampling.